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Using Innovative Tools to Teach Computer Application to Business 
Students - A Hawthorne Effect or Successful Implementation Here to Stay 


Abstract 

A year after the primary study that tested the impact of introducing blended learning and guided discovery to 
help teach computer application to business students, this paper looks into the continued success of using 
guided discovery and blended learning with learning management system in and out of classrooms to enhance 
student learning. Particularly it looks at the successful adaptation and implementation of learning 
management system as an aid to learning and delivery of a computer application subject to a class that is 
majorly populated by business students with little or no IT background. 

This paper tests the continued success to establish whether the success recorded after the first implementation 
was sustainable or merely influenced by the Hawthorn effect of trialing an innovation. It concludes with final 
grade measure over a one-year period of implementation of the system, and student responses that strongly 
support the original approach used by the author. 
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Introduction 

I always tended to pay more attention to studies before a test when I knew Dad was watching and 
scored better in those tests than others. 

- Peter Hitch, 22, 
UOWD Graduate 


Learning is a process that is meant to teach students concepts and skills, and eventually the ability 
to apply what they learn to life outside classrooms. However, educators around the globe seem at a 
loss for an answer to the dilemma of the right method of teaching that will ensure the right type of 
learning, particularly when teaching a subject from one stream to students in another. A number of 
factors, such as the language used, methods of teaching, pedagogical fit and the textbooks used, 
contribute to the success or failure of teaching a subject. 

With the advancements of information communication technology, the impact on academia has 
been well-documented over the past decades (Bransford, et al. 1999; Hepp et al. 2004; Kozma 
2005; Unwin 2009; Anderson 2009; Resta & Patru 2010), as have the ways in which these 
advancements have pushed learning beyond the classroom walls (Anderson 2010). 

Some researchers have suggested that this technology has made teaching easier for academics in 
the 21st century, giving academics tools to enhance their teaching styles, strategies, 
communications and classroom environments (Collis 1999; Kozma & Anderson 2002; UNESCO 
2010). In keeping with these studies, in January 2011 the author introduced a learning- 
management system coupled with guided discovery that allowed business students with little or no 
experience with computers to better understand computer applications; this project aimed to 
reduce failure rates for the subject (Khan 2011). While the results were highly satisfactory, with 
lower failure rates, in teaching computer applications to business students increased student 
engagement and overall student satisfaction, questions arose almost instantly regarding the long¬ 
term sustainability of the tools and pedagogy applied. Given that the results were based on only 
one semester's observations, they may have been influenced by the Hawthorne effect, in which 
people change their behaviour when they are under observation. The Hawthorne effect was first 
documented in 1955 when a researcher analysed data from a factory in Hawthorne to understand 
the impact of lighting levels within the building on workers' productivity, and concluded that the 
factory workers were more productive not because of the change in lighting but because they were 
being watched (Gale 2004). 

Although much has been written for and against the Hawthorne effect, the purpose of this paper is 
not to confirm or refute it, but merely to establish the primary study’s conclusions as sustainable. 
The purpose of the current study is to follow Shayer (1992) in revisiting the original study that 
was conducted in the spring semester of 2011 at the University of Wollongong in Dubai, and 
extrapolate it to three more semesters that together will comprise a control semester. The paper 
studies grades from classes taught by faculty members other than the author to test the results from 
the primary study. The current study also aimed to establish the long-term sustainability of using 
learning-management systems and guided discovery beyond the scope of a test environment to 
enhance the teaching of computer subjects to business students. 
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University of Wollongong in Dubai 

The University of Wollongong in Dubai (UOWD) is a direct campus of the University of 
Wollongong in New South Wales, Australia. UOWD started 18 years ago in Dubai, a business 
hub of the United Arab Emirates (UAE). Due to its early entry into the higher-education market, 
the university systematically positioned itself as offering a Western education in the Gulf. It is also 
fully accredited by the Ministry of Higher Education and therefore is able to offer its graduates 
either an Australian degree or a UOWD degree, both of which are recognised in the UAE job 
market. 

UOWD primarily offers two degree streams: computers and engineering, and business and 
finance. Currently it offers undergraduate, postgraduate and PhD programs in a host of degree 
majors for over 3,500 students from over 100 nations, on its campus at Dubai’s Knowledge 
Village. 

All undergraduate students, regardless of the degree they are pursuing, must comply with the 
accreditation policy in the UAE, which stipulates that they must enrol in general-education courses 
in their first semester. One of these required subjects is Computer Applications; it has an average 
class size each semester of over 100 students. 

Teaching computer applications to business students 

According to a study carried out by the Business Education Statewide Advisory Committee in 
California, USA, teaching computer-application subjects to students with non-computer-related 
backgrounds such as business creates problems because of the range of ability within the group. 
For this reason, the computer-applications teacher must be prepared to present material in a variety 
of ways, and to adjust to the students’ knowledge of computers as well as their individual learning 
styles (BESAC 2011). UOWD instructors also face these circumstances. The author has been 
teaching the subject since 2006 and sometimes faces a class of over 300 students. 

2.1.1 Student population and demography 

Most of students in this subject - usually about 85% - are from either the Bachelor of Business 
Administration or the Bachelor of Commerce. Almost all are just out of high school, and range 
from 16 to 19 years old. They come having studied different school curricula, with varying levels 
of English (not as a first language) and different experiences of class-attendance policies and 
supervised learning. 

2.1.2 Issues with teaching computer applications to business students 

Earlier studies assert that because technology is often readily available to students at home, in 
libraries or on mobile devices, it has created a challenge for teachers because of the range of 
abilities and skills within the cohort (Khan 2011; Lundgren, Lundgren and Mundrake 1995). This 
is also true for students enrolling in the Computer Applications course at UOWD. Some students 
have very little exposure and experience while others may be creating their own apps for their 
mobile phones, making them immensely tech-savvy, although not necessary tech-aware (Lundgren 
et al. 1995). 

Another issue is the subject itself. Computer Applications at UOWD covers the following topics: 
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• Application software 

• Operating systems 

• Central processing systems 

• Input and output devices 

• Storage and multimedia 

• Networking and computer connectivity 

• Internet 

• Social computing 

Students are also taught to use Word, Excel, Access, Front Page, PowerPoint and Movie Maker. 
Although the topics sound simple, students have often commented in subject evaluations that the 
depth of information is “overwhelming” or “too difficult”, that it “[makes] the subject less 
interesting” and that they are “not sure what it all means” (Office of Institutional Effectiveness 
2007-2010). 

As attendance is not compulsory in lectures, and as many students show low interest levels due to 
the difficulties posed by the depth of the subject or the English language barrier, the failure rate for 
the subject has been consistently high, at about 14% every semester (Subject Report 2007-2011). 

Guided discovery and blended learning 

Jerome Bruner (1967), John Dewey (1997), Jean Piaget (1954) and Lev Vygotsky (1978), suggest 
that discovery learning encourages students to become active participants in the learning process 
as they explore concepts and answer questions through experience. This student-centred practice 
covers an instructional model and strategies that focus on active, hands-on learning opportunities 
for students, as outlined by Dewey (1997) and Piaget (1954). It has become important to 
understand students’ minds, how they learn, what piques their curiosity, what catches their eye. 
According to Grantham in an overview of student learning: ‘learning is a change in [students’] 
conceptions - a change in [their] understanding" it has been critical to understand that "students 
have their own way of learning, a ‘learning style’, and they will learn best when there is a variety 
of learning opportunities that give them their chance to learn in their own way" (Grantham 2005). 

Based on the literature, and given the student demography and the issues arising from their 
varying backgrounds, the author decided to adopt guided-discovery practice to teach computer 
applications with technology integration, given the increase in the use of technology in students’ 
personal lives. The author chose to use teaching methods that would compete with those forms of 
entertainment that hold students' interest, such as Facebook (Khan & Samuel 2009). 

There is also a perception that “Students never learn from technology per se; they learn from the 
strategies teachers use to communicate effectively through the technologies” (Singh and Reed 
2003). This is true for all tools of learning. The author researched the practices across faculties, 
finding that the concept of blended learning has been in practice at UOWD for some time, with 
teachers using intranet websites, traditional classroom settings, textbooks, roleplay and other 
methods. 

The author decided to apply blended learning by incorporating a learning-management system to 
the subject with several objectives: 
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• to increase student involvement with subject, peer and lecturer; 

• to encourage independent learning in students, giving them flexibility in scheduling their 
studies and taking their assessments; 

• to help students develop strategies to overcome language barriers; and 

• to extend students’ learning experiences beyond the traditional classroom and textbook 
reading. 

4. Primary study 

The original study (Khan 2011) had been carried out in the spring semester (January 2011-April 
2011) at UOWD. The author used a learning-management system (LMS) to embed videos and set 
up online quizzes and student forums. The aim was to engage students with the subject, and draw 
their attention to the content through innovative learning strategies such as interaction in the 
forums. By using the LMS, the author managed to significantly reduce student-accessibility 
problems, as now students could access the notes and assessments and participate from anywhere 
and at any time. 

The results highlighted the benefits of using an LMS (Khan 2011): 

• the online tests helped increase frequency of tests and removed the need for student 
attendance at the tests; this reduced the failure rate from 14% in the autumn semester to 
6% in the spring semester. Given that no other factors were changed, the reduction was 
attributed to the introduction of the blended -learning tool that allowed for guided- 
discovery learning for students, and to the frequency of quizzes, which may have helped 
students stay focused and study more. 

• Online discussions involved students with the topics so that all of the students who 
participated in the discussions correctly answered the questions in the final exam. 

• Students developed independent learning at their own pace and location, ensuring that 
attendance was not a barrier to learning. 

• Because the LMS was coupled with the traditional classroom setting, students could learn 
at their own pace while connecting the topics discussed in class to real-life scenarios. As 
there was a huge dependence on technology, students were more engaged with the 
subject, taking the time to understand the language and follow up by attending the next 
sessions to keep abreast of the topics covered. 

A Hawthorne effect, or a successful, sustainable implementation? 

The primary study demonstrated the adaptation and implementation of the guided-discovery style 
along with a blended learning-management system to attain the following objectives (Khan 2011): 

• increase student involvement with subject, peer and lecturer; 

• encourage independent learning in students, giving them flexibility in scheduling their 
studies and taking their assessments; 

• help students develop strategies to overcome language barriers, to extend their learning 
experiences beyond traditional methods; and’ 

• allow business students a better learning environment to grasp computer-application 
concepts, thereby reducing the failure rate in the subject 
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It was concluded that innovative learning practices, if structured and aided by blended learning 
tools, could enhance business students’ learning computer applications at UOWD and reduce 
overall failure rates in the subject. 

According to Gale (2004) the Hawthorne’s effect is demonstrated when people change their 
behaviour because they are being watched. The celebrated studies of employee behavior at the 
Hawthorne Plant of the Western Electric Company between 1920 and 1930 gave rise to a 
subsequent by-product: the Hawthorne effect. Despite the puzzlement caused in earlier research 
there is still not one accepted definition for the term (Draper 2002). For the purpose of this paper, 
the author identifies it as: When watched, a subject will automatically act differently, hence 
altering the course of the study. 

However, disputes in the academic world have raged for years as the Hawthorne Effect has often 
been misguidedly used as an " [ejxocet in order to invalidate classroom-based research" (Coombs 
& Smith 2003 p99). Where academics have tried to use classroom settings to "determine if a new 
teaching method or curricular improvement might be superior to an on-going pattern of teaching 
or curriculum" (Cook & King 1968), others have questioned its authenticity (Cook 1967). 

There already exists a body of literature that has looked at both sides of the argument, some stating 
that there needs to be a control that dictates whether the experiment yields true results and the 
researcher is an impartial observer, and others stating an expectation that researcher and his or her 
subjects should explicitly work together to produce results (Adair 1984; Broches 2008; Carey 
1967; Diaper 1990; Drew, Hardman & Hart 1996; Gale 2004; Franke & Kaul 1978; Gransberg & 
Holmberg 1992; Jones 1992; Kuhn 1996; Kolata 1998; McQuarrie 2005; Rice 1992). 

A year after the initial study, this follow-up study was designed to test whether the failure rates 
remained low with the use of guided discovery and the LMS. 

The proposed hypothesis was: 

The reduction in failure rate found in the primary study is a sustained effect of implementing 
guided discovery and blended learning. 

Methodology 

The primary study was carried out during the spring semester 2011 The subsequent study was 
conducted during summer 2011, autumn 2011 and spring 2012 (completing a year of study). 

Summer 2011 was chosen as a "control", where neither guided discovery nor the LMS was used. 

A control was introduced primarily for two reasons (adapted from Little 2010): 

(1) to check if the system was behaving as expected from previous knowledge, experience and 
observations; and 

(2) to make it possible to pinpoint any errors in the experiment so they could be corrected for the 
next implementation. 

Although the application of controls to experiments originated in medical research, the literature 
has shown that the reactive effects arising from conducting drug research were also encountered in 
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behavioral research such as this study (Adair, Sharpe & Huynh 1999); hence a control was applied 
to the current study. 

As spring 2011 was the first semester to implement the LMS, students were informed during the 
semester that the system was on a trial basis and that they would be observed and recorded. 
However, the students in the subsequent semesters - autumn 2011, summer 2011 and spring 2012 
- were not informed, as the LMS was now considered a part of the subject on offer. The author 
applied the same realigned subject outline from spring 2011 to autumn 2011 and spring 2012 to 
include the following LMS tools: 

• Lecture notes and embedded videos 

• Graded online discussions 

• Quizzes 

• Student forums 

It is imperative to note that the current study included three other faculty members besides the 
primary author as lecturers and teaching assistants in summer 2011, autumn 2011 and spring 2012. 
In keeping with Shayer (1992), following three stages were implemented to try to negate the 
Hawthorne effect, if it existed: 

• Stage 1: the initial study that highlighted the primary effect (Khan 2011). 

• Stage 2: replication - could it be done by others besides the researcher (current study 
using other faculty members to teach the subject along with the author, using the student 
manual developed during the previous study)? 

• Stage 3: general application - could any non-enthusiast be trained to apply the same 
tools? To assist other faculty members in using the LMS and guided discovery, the 
author developed an Instructors’ Manual and held two separate workshops during autumn 
2011, training others to proceed with the learning tool and associated pedagogy, 
particularly in autumn 2011 and spring 2012. 

At the end of each semester, the grades were recorded along with student evaluations gathered by 
the university’s quality-assurance department. 

7. Results and discussion 

The combination of various LMS tools and guided discovery continued to bring positive feedback 
from students and reduce failure rates. 

Despite the involvement of more than one faculty member, the online tests proved to continually 
reduce administrative time for those who taught the subject over the autumn 2011 and spring 2012 
semesters. This allowed the author and other faculty members to increase the frequency of quizzes 
and devote more time to helping students during the semester, particularly during autumn 2011, 
with a class size of 262. 

As the frequency of tests increased and students got more involved with the subject, the failure 
rate decreased further. As illustrated in Figure 2, the failure rate in spring 2011 semester was the 
lowest recorded between 2007 and 2011, at 6.4%; autumn 2011 was 5.73% and spring 2012 
(based on mock exams) was 5%. Given that no other factors were changed, the continued 
reduction was attributed to the use of the blended-learning tool that allowed for guided discovery 
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for students, and to the frequency of quizzes that helped students stay focused, study more and be 
more engaged with the subject. 

However, the control semester, summer 2011, saw a spike in failure rate to 34.38%, providing 
further evidence that not using guided discovery or a learning-management system definitely 
hindered student learning. 



Figure 1: Failure-rate data collected from subject assessment meetings at UOWD between 2007 and 2012 


Additionally, 

• Over 1,200 posts were made in the course of autumn 2011 and spring 2012 semesters, 
which gave an indication of student participation. 

• Students corresponded with each other and lecturer(s); any issues or doubts were 
clarified during the semester. 

According to student feedback, the LMS coupled with traditional classroom settings allowed 

students to: 

• choose their own pace while learning; 

• understand and connect the topics discussed in class to real-life scenarios, giving them a 
direction to the subject and familiarity; and 

• be more engaged with the subject. As there was a huge dependence on technology,, 
students took time to understand the language and follow up by attending the next 
sessions to keep abreast of the topics covered 

(Office of Institutional Effectiveness, Autumn 2011). 

Among other results (Figure 3), 

• over 50% of respondents used the LMS a few times a week in both the semesters - 
showing considerable frequency of usage that increased from spring 2011; 

• 65% of the respondents accessed the system from home, and 78% from UOWD computer 
labs, among other locations, showing a variety of geographically separated locations as 
points of access that students found to be highly convenient, similar to spring 2011; 

• about 80% - an increase from spring 2011 - agreed or strongly agreed that they were 
comfortable with the LMS, using it from home, work or UOWD, that the instructor 
adequately introduced the system in the classroom and that it was easy to get started on 
the system; and 
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• over 70% - also an increase from spring 2011 - agreed that using the LMS improved 

their communication with the instructor and other students; gave more access to resources 
and learning materials; made them feel more connected to the subject; and made it easier 
for them to learn, supporting the overall outcomes and perceived objectives of using the 
learning tool. 



Figure 2: Summarised responses from respondents on LMS use and effectiveness 

Finally, the author conducted a survey of the faculty members who had used the Instructors’ 
Manual to determine its effectiveness (Figure 4). 


Strongly agree 
Ag ree 

Neither Agree 
nor Disagree 

L Di ;agree 


Figure 3: Response to LMS Manual and usage 


As illustrated in Figures 2 and 3, the faculty members used the manual effectively to enhance their 
ability to use the LMS and guided discovery. The subsequent impact of their increased ability and 
knowledge has already been discussed in the form of reduced failure rates in autumn 2011 and 
spring 2012. 


8. Conclusion 


Understanding student needs to enhance their learning experience has been the driving force 
behind major shifts in teaching paradigms in the last few decades. The process is continuous, ever- 
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growing, ever-evolving. Teachers are trying their utmost, competing against the odds to enhance 
student learning through student-centered learning styles without the added pressure of facing 
criticism for teaching practices and pedagogy that they try out in classroom settings. 

Literature has revealed that though the Hawthorne effect may not be established, and may not even 
have a confirmed and recognised definition, it continues to be a bane for academic researchers as 
they introduce evolving teaching practices through action research (Coombs and Smith 2003). 

This paper looked closely at ways to combat the Hawthorne effect and accomplish the long-term 
sustainability of the results of the initial study by Khan (2011). The current study looked at 
introducing a control to better understand the impact of introducing LMS and guided discovery to 
aid in teaching computer applications to business students. This subsequently enhanced 
understanding, in that the control semester, when instructors did not use the proposed tools, 
showed a dramatically increased failure rate. The study also looked at introducing the three stages 
of research combat by Shayer (1992). 

• The first stage (that of an actual study being conducted) was completed in 2011 over a 
period of one semester, testing the impact of introducing the LMS and guided discovery 
on failure rates. 

• Stages two (replicating test with enthusiasts) and three (generalising to train others) were 
conducted over a period of two more semesters, besides the control. Both stages 
successfully presented lowered failure rates, indicating that the LMS tools and guided 
discovery were indeed effective tools to teach computer applications to business students, 
further reducing the likelihood of a Hawthorne effect. 

Hence it is concluded that innovative learning practices, if structured and aided by blended 
learning tools, can enhance business students’ learning in the Computer Applications subject at 
UOWD, and reduce overall failure rates in the subject. Moreover, this reduction can be achieved 
not just in the initial stages of the research, but sustainably, as the result of a cascading process to 
ensure lower failure rates for all semesters, all student populations, taught by any faculty 
following the proposed tools and pedagogy. 

Draper (2010) provides food for thought for a possible area for future study: 

[Iff what you care about is improving learning and the learners' experience, then you 
may want to maximize not avoid halo and Hawthorne effects. If you can improve learning by 
changing things every year, telling students this is the latest thing, then that is the ethical and 
practical and practically effective thing to do. 
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